Combining Formal and Distributional Models of Temporal and Intensional Semantics

We outline a vision for computational semantics in which formal compositional semantics is combined with a powerful, structured lexical semantics derived from distributional statistics. We consider how existing work (Lewis and Steedman, 2013) could be extended with a much richer lexical semantics using recent techniques for modelling processes (Scaria et al., 2013)—for example, learning that visiting events start with arriving and end with leaving. We show how to closely integrate this information with theories of formal semantics, allowing complex compositional inferences such as is visiting→has arrived in but will leave, which requires interpreting both the function and content words. This will allow machine reading systems to understand not just what has happened, but when. 1 Combined Distributional and Logical Semantics Distributional semantics aims to induce the meaning of language from unlabelled text. Traditional approaches to distributional semantics have represented semantics in vector spaces (Baroni et al., 2013). Words are assigned vectors based on collocations in large corpora, and then these vectors a composed into vectors representing longer utterances. However, so far there is relatively limited empirical evidence that composed vectors provide useful representations for whole sentences, and it is unclear how to represent logical operators (such as universal quantifiers) in a vector space. While future breakthroughs may overcome these limitations, there are already well developed solutions in the formal semantics literature using logical representations. On the other hand, standard formal semantic approaches such as Bos (2008) have found that hand-built ontologies such as WordNet (Miller, 1995) provide an insufficient model of lexical semantics, leading to low recall on applications. The complementary strengths and weaknesses of formal and distributional semantics motivate combining them into a single model. In Lewis and Steedman (2013), we proposed a solution to these problems which uses CCG (Steedman, 2012) as a model of formal semantics, making it straightforward to build wide-coverage logical forms. Hand built representations are added for a small number of function words such as negatives and quantifiers—but the lexical semantics is represented by first clustering predicates (based on their usage in large corpora), and then using the cluster-identifiers as symbols in the logical form. For example, the induced CCG lexicon might contain entries such as the following1: write ` (S\NP)/NP : λyλxλe.rel43(x, y, e) author `N/PPof : λyλxλe.rel43(x, y, e) Equivalent sentences like Shakespeare wrote Macbeth and Shakespeare is the author of Macbeth can then both be mapped to a rel43(shakespeare,macbeth) logical form, using derivations such as: Shakespeare wrote Macbeth NP (S\NP)/NP NP shakespeare λyλxλe.rel43(x, y, e) macbeth > S\NP λxλe.rel43(x,macbeth, e) < S λe.rel43(shakespeare,macbeth, e) This approach interacts seamlessly with standard formal semantics—for example modelling negation by mapping Francis Bacon didn’t write Macbeth to ¬rel43(francis bacon,macbeth). Their method has shown good performance on a dataset of multi-sentence textual inference problems involving quantifiers, by using first-order theThe e variables are Davidsonian event variables.